Identi cation of Non - linear Systems using Empiri - cal Data and Prior Knowledge { An

The choice of a parametric model structure in empirical and semi-empirical non-linear modeling is usually viewed as an important and critical step. However, it is known that by augmenting the least-squares identiication criterion with a term that imposes penalty on the non-smoothness of the model, an optimal non-parametric model can be found explicitly. The optimal non-parametric model will depend on the particular form of the penalty, which can be looked upon as a priori knowledge, or the desired properties of the model. In this paper these results are extended in several directions, i) we show how useful types of prior knowledge other than smoothness can be included as a term in the criterion or as a constraint, and how this innuences the optimal model, ii) dynamic models and a general prediction error penalty are considered, iii) we present a practical numerical procedure for the identiication of a close to optimal semi-parametric model. The numerical approach is motivated by the diiculty of deriving the optimal non-parametric model if there are complicated constraints or penalty terms in the criterion. Finally, iv) we discuss determination of the appropriate model complexity through selecting weights on the diierent penalties on the basis of empirical data. Since the optimal non-parametric model is identiied using an augmented optimization criterion, and all prior knowledge and desired model properties may be speciied through the augmented optimization criterion, it is the choice of this criterion that is critical in this approach. This elevates the empirical and semi-empirical modeling problems to a more transparent and engineering-friendly level than is achieved by directly specifying a parametric model structure. Hence, our results shed some more light on modern non-linear empirical modeling approaches like radial basis-functions, splines and neural networks. In addition to providing a fundamental insight into the role of diierent kinds of prior knowledge, this high-level formulation is also attractive from a practical point of view, as it lies closer to engineering thinking, and less guesswork is required. The exibility and power of this approach is illustrated with a semi-realistic simulation example.